Collaborative Learning of Hierarchical Task Networks from Demonstration and Instruction

We present an approach for learning complex procedural tasks from human demonstration. Our main innovation is to view the interaction between the human and the robot as a mixed-initiative collaboration. Our work utilizes hierarchical task networks to enable the robot to learn complex procedural tasks. Our contribution is to integrate hierarchical task networks and collaborative discourse theory into the learning from demonstration paradigm to enable robots to learn complex tasks in collaboration with the human teacher. We demonstrate techniques for generalization based on merging multiple demonstrations and policy refinement through question asking. We validate our approach in a car maintenance domain.